Using Bayesian filtering to interpret tactile data during flexible materials manipulation

Localizing and manipulating features such as buttons, snaps, or grommets embedded in fabrics and other flexible materials is a difficult robotics problem. Approaches that rely too much on sensing and localization that occurs before touching the material are likely to fail because the flexible material can move when the robot actually makes contact. This paper experimentally explores the possibility of using proprioceptive and load-based tactile information to localize features embedded in flexible materials during robot manipulation. In our experiments, Robonaut 2, a robot with human-like hands and arms, uses particle filtering to localize features based on proprioceptive and tactile measurements. Our main contribution is to propose a method of interacting with flexible materials that reduces the state space of the interaction by forcing the material to comply in repeatable ways. Measurements are matched to a “haptic map”, created during a training phase, that describes expected measurements as a low-dimensional function of state. We evaluate localization performance when using proprioceptive information alone and when tactile data is also available. The two types of measurements are shown to contain complementary information. We find that the tactile measurement model is critical to localization performance and propose a series of models that offer increasingly better accuracy. Finally, the paper explores this localization approach in the context of two flexible materials insertion tasks that are important in manufacturing applications.